Repository navigation
docs: list the AI Gateway models that return array chat content - #6789
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Part of one change with: |
andrelandgraf
left a comment
There was a problem hiding this comment.
Aileen review: approve (fix, low risk, 49c0cd0)
Verdict
Approve at low risk. The change corrects the conformance record for two Claude models and the docs that describe it. Nothing touches auth, billing, migrations, or secrets.
What I verified
- Live probe: I created a throwaway project in the Testing smoke org, probed
/v1/chat/completions, then deleted the project.- On a reasoning prompt,
claude-sonnet-5-5,claude-fable-5-1,claude-haiku-5-5, andclaude-opus-5-5returnedarray[reasoning,text]. claude-sonnet-5returned a string. That matches the$howToRegeneratecorrection.gemini-3-8-flashreturnedarray[text], andgpt-5-minireturned a string.- The easier bat-and-ball prompt got strings from every Claude model. This confirms that the record has to be measured with a reasoning prompt.
- On a reasoning prompt,
- Streaming:
claude-sonnet-5-5sent 2 array reasoning chunks and 120 string text chunks.gemini-4-argonsent string chunks and one finalnull. The new string-only streaming snippet keeps the full answer for both. - Provider claim: the claim that
@neon/ai-sdk-providerreturns plain text holds in neon-pkgs code. Claude routes to native Messages, and Gemini chat goes throughwrapFetchWithHarmonyNormalization. - Effect on
/models:claude-sonnet-5-5andclaude-fable-5-1now get the same examples asclaude-opus-5-5: AI SDK, Mastra, and Anthropic Messages cURL. They lose the OpenAI SDK TypeScript and Python snippets. Those snippets break once the model reasons, so this is intended. The.mdmirrors are gitignored and generated at build time, so nothing needed regenerating. - Factory checks:
prettier --checkpasses on the 5 changed content files.- eslint reports one
import/orderwarning inroute.test.js. It is also present on main. npm run test:unit:run -- --retry=2 --maxWorkers=4 --testTimeout=30000 --hookTimeout=60000passed: 106 files, 1463 tests.verify-agent-endpointspassed 68/68 checks, andcheck:models-endpoint-syncreports 63 models in sync.check:docs-api-consistencyfound no broken references.
Non-blocking nits
From me and a second-opinion Claude Opus 5.5 review:
-
chat-completions.md:161says "Every other model returns a string".models.mdsays "Every other chat model", which is more precise. -
The Gemini streaming sentence in
models.mdleaves out the final-chunknull. The snippet already handles it. -
content-shape-docs.test.js:- It ignores backticked ids that are missing from
capabilities.json. - It would fail if a conforming model were named inside the admonition.
- Its second test only checks link wording.
This is acceptable for a guard on hand-maintained prose.
- It ignores backticked ids that are missing from
-
Already listed in the PR's "Left open" section:
- The console's conformance record in neondatabase/neon-console-code-examples still lists both models as
conforms. - The reason the Mastra example gives for using
neon(...)is out of date. - The root cause is in the gateway.
- The console's conformance record in neondatabase/neon-console-code-examples still lists both models as
|
Waiting for Andre: category fix needs Andre. Merge it when it is ready, or comment to have Aileen change it. |
…auge-6c1bc1f98584
…vior Drop claude-sonnet-5-5 (returns a string), add gemini-4-carbon and claude-fable-5-2, show both array shapes, and replace the OpenAI SDK snippets with one helper that joins text blocks for messages and streamed deltas.
Problem
On
/v1/chat/completions, Neon AI Gateway returnsmessage.contentas an array of content blocks for some Claude and Gemini models instead of the string the OpenAI Chat Completions spec requires. It's a known gateway issue that's being fixed.The "Content shape varies by model" warning in the AI Gateway models doc named only
claude-opus-5-5andgemini-4-argon, then told every reader to handle both shapes. Several models that return arrays weren't listed. The chat completions page, the get-started "Swap models" section and two FAQ pages repeated the blanket advice, and get-started said swapping models needs "no other code changes"./modelsand the per-model docs pages pick their code examples fromsrc/app/models/capabilities.json. That record listedclaude-fable-5-1asconforms, so its page served plain OpenAI SDK snippets that readmessage.contentas a string. Those snippets break once the model reasons.What the gateway returns
Measured live against
/v1/chat/completions:message.contentdelta.contentgemini-3-7-flash,gemini-3-8-flash,gemini-4-argon,gemini-4-carbonnullon the final chunkclaude-opus-5-5,claude-haiku-5-5,claude-fable-5-1,claude-fable-5-2claude-sonnet-5-5,gpt-5-miniThe array shapes:
The docs change
The warning in
content/docs/ai-gateway/models.mdnow lists the eight models above, says which return arrays on every call and which only once they reason, shows both shapes and states that this is a known gateway issue being fixed. It points readers to@neon/ai-sdk-providerfor plain text from every model, and to the native Gemini and Anthropic Messages endpoints, which return a fixed shape.For OpenAI SDK users, one helper replaces the old pair of snippets. It joins the
textblocks and reads messages and streamed deltas the same way:The old snippet took only the first
textblock and the old streaming snippet dropped every array delta. The helper keeps all text in both cases.chat-completions.md,get-started.mdand the two FAQ pages now say "a few Claude and Gemini models" and link to the list. Get-started says no other code changes are needed "for most models".Also in here
src/app/models/capabilities.json:claude-fable-5-1isarray-content, so/modelsand its docs page serve the Anthropic Messages cURL example instead of OpenAI SDK snippets. The$commentand$howToRegeneratenotes name the Claude models that return arrays once they reason, and say to update the docs list when the record changes.src/app/models/content-shape-docs.test.js(new): fails when the warning's model list disagrees with thearray-contententries incapabilities.json, or when it stops linking the provider and both native endpoint pages.src/app/models/route.test.js:claude-fable-5-1gets no OpenAI SDK chat examples, and its cURL example uses/anthropic/v1/messages.src/app/models/examples.js: a comment dropped a hard-coded model count that no longer held.Verification
textOfhelper type-checks undertsc --strictand returns the right text for a string,null, the Gemini array, the Claude reasoning-plus-text array and a reasoning-only array.npm run test:unit:runpasses, including the new tests.npm run check:docsandnpm run check:docs-api-consistencypass.check:mdandlint:mdruns report failures only in files this PR doesn't touch./docs/ai-gateway/models.mdmirror, which is generated at build time.For your attention
gemini-4-carbonandclaude-fable-5-2aren't incapabilities.json, so/modelsdoesn't serve them and their docs pages show no code examples. Adding them needs measured values for every field (native dialect, Responses, web search, image generation), and that's a separate change. Until then, the new test can't check those two entries in the docs list.claude-fable-5-1asconforms. The nextbun run probe:conformancerun there needs a reasoning prompt to pick it up.array-contententries can be removed.